perm filename ISI.CL[COM,LSP] blob sn#819595 filedate 1986-06-22 generic text, type C, neo UTF8
COMMENT āŠ—   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	āˆ‚20-Jun-86  1243	berman@isi-vaxa.ARPA 	CL support   
C00056 ENDMK
CāŠ—;
āˆ‚20-Jun-86  1243	berman@isi-vaxa.ARPA 	CL support   
Received: from ISI-VAXA.ARPA by SU-AI.ARPA with TCP; 20 Jun 86  12:41:59 PDT
Received: by isi-vaxa.ARPA (4.12/4.7)
	id AA16157; Fri, 20 Jun 86 12:41:00 pdt
From: berman@isi-vaxa.ARPA (Richard Berman)
Message-Id: <8606201941.AA16157@isi-vaxa.ARPA>
Date: 20 Jun 1986 1240-PDT (Friday)
To: RPG@SU-AI.ARPA
Cc: berman@isi-vaxa.ARPA
Subject: CL support


Here's ISI's proposal showing our plans to support the CL community:

                              COMMON LISP SUPPORT

1. Background
  The effort by the Common Lisp community to define and propagate a Common Lisp
Standard is well under way.    At  the  just  concluded  Common  Lisp  Meeting,
agreement  was  reached  to pursue such standardization under ISO.  Bob Mathis,
who helped guide the ADA standardization and validation efforts from  his  post
as  head  of  AJPO,  has  been  chosen as the convener of an expanded technical
committee to address the remaining technical issues in defining the Common Lisp
Standard and to resolve technical questions relating to adopted portions of the
standard.

  However, the benefits arising from such standardization will not be  realized
unless  DARPA  builds  the  infrastructure  needed  to  nurture and support the
emerging Common  Lisp  community.    The  Common  Lisp  specification  must  be
correctly  maintained  and  distributed  to  those  who  have  need  of  it.  A
validation suite is needed to guide developers  toward  proper  implementations
and  to  ensure  that  they have correctly done so.  A library of public domain
information concerning Common  Lisp  (validation  suites,  useful  utility  and
functional programs, documentation, tests, implementation guides, etc.) must be
maintained and disseminated.  Past and  future  proceedings  of  the  Executive
Committee  must  be  archived  and  organized  for retrieval by date and topic.
Libraries of public domain  information  concerning  Common  Lisp  (source  and
object  code,  documentation,  tests,  implementation  guides,  etc.)  must  be
maintained and disseminated.  Network mailing lists of people and organizations
participating   in   the   electronic  forums  used  to  raise  issues,  submit
suggestions, and arrive at consensus must be maintained.  The messages must  be
organized  and summarized so that new people can join the forum and participate
in the distributed standardization effort.  Finally, travel and  administrative
support is needed for the convener of the Common Lisp standardization effort.

  The  Common Lisp community needs a technically competent support organization
to provide these services.  Furthermore, this support group must have no  stake
in  any  Common Lisp implementation so that they can perform the necessary work
with complete impartiality.  ISI recognizes the need for these services and has
the   technical  and  administrative  expertise  to  support  the  Common  Lisp
community.  Moreover, ISI has the trust of that community.    ISI  is  uniquely
qualified  to  support  this  effort  because of its personnel, long history of
service support of the Lisp community, and demonstrated ability to  manage  and
operate  service  efforts.  Bob Mathis was chosen as the convener by the Common
Lisp community.  Ron Ohlander was chosen to be a  member  of  the  Common  Lisp
Executive  Board.    Richard  Berman  has formed working relationships with the
validation experts in the vendor organizations.  All of these  people  are  ISI
employees.    ISI  has  established  good  relations with many of these vendors
through its negotiation,  purchase,  and  distribution  of  Lisp  machines  for
DARPA's  Strategic  Computing program.  Prior to that, ISI produced, at DARPA's
request, a full compatible Interlisp for the Vax to promote its widespread use.
Several   hundred  copies  have  been  distributed  and  the  system  has  been
transferred  to  DEC  for  further  distribution  and  maintenance.    ISI  has
successfully operated and managed the MOSIS service, a TOPS-20 resource center,
and remote support  for  DARPA's  computing  services.    This  combination  of
required capabilities makes ISI uniquely qualified for this task.

2. Tasks



2.1. Common Lisp Validation
  A  public  domain  validation suite is desperately needed.  Several purported
implementations of Common Lisp exist or will exist within the next few  months.
The  community  has  no way of determining the degree of compliance obtained by
such  implementations.    A  few  companies  have  extensive   (though   hardly
comprehensive)  validation  suites,  but all these are proprietary.  Building a
public  domain  validation  suite  would  be  quite  expensive,  would  require
expertise  in many different areas, and would take any one organization quite a
while to produce (because of staffing and phasing issues).

ISI proposes to pursue a very different approach.  At the just concluded Common
Lisp  Meeting,  we chaired a discussion of the problems concerned with creating
such a public domain validation  suite  and  investigated  the  alternative  of
creating  a public domain validation suite from vendor contributions.  We found
much support for this alternative among the vendors if the resulting validation
suite  would  be  public  domain  and  if  it was collected and maintained by a
technically competent, neutral, and non-commercial organization  such  as  ISI.
This  support  was quite widespread and included both the vendors with existing
extensive proprietary validation suites and those who had not yet created  such
proprietary  suites.  The former agreed to contribute their existing suite, and
the latter agreed to produce and contribute one for some portion of the  Common
Lisp  Standard.  While  these  "agreements" are merely expressions of intent at
this point, we feel that they are based on a true sense of  the  needs  of  the
emerging  Common Lisp community and the benefits that can accrue to the members
of that community. We have been working with these vendors  to  solidify  these
"agreements" into hard commitments and have just begun to receive contributions
from these vendors.  For information purposes, we have  attached  the  list  of
vendors  that  have  informally  agreed to contribute to a public domain Common
Lisp Validation Suite (see Attachment A).

ISI's role would differ for the two types of contributions.  For  the  existing
suites, ISI must:

   1. Homogenize  the  tests so that they are included in a common way and
      report their results in a common way  (currently,  each  vendor  has
      their own conventions).

   2. Eliminate duplicate tests.

   3. Determine  the  areas  of coverage and the degree of coverage within
      those areas.


For the newly produced suites, ISI must:

   1. Specify the way that tests will be invoked and report their results.

   2. Coordinate the focus of the vendors to maximize coverage.


In addition, ISI must:

   1. Build  a  Validation  Manager  to  invoke  and  collect  results  of
      individual tests.

   2. Determine the validity of each contributed test.

         a. Correct any incorrect ones,

         b. Forward   questionable  ones  to  the  Common  Lisp  Technical
            Committee for resolution of ambiguity.

   3. Catalog each contributed  test,  identify  its  contributor(s),  and
      place it in an appropriate area of the validation suite.

   4. Build  and  maintain  a  Public  Domain Validation Suite library and
      provide access to it.


Finally, ISI must conduct the validation of vendor's implementation.  This will
be  done  by  ISI  personnel visiting the vendor site.  The vendor will provide
visiting validation team access  to  the  software  to  be  validated  and  the
hardware  upon  which  it  runs.    The ISI validation team will merely run the
validation suite, collect its results, and  submit  them  to  the  Common  Lisp
Steering  Committee.  The Committee will evaluate the results and determine the
degree to which the vendor's  implementation  complies  with  the  Common  Lisp
Standard.



2.2. Library of Public Domain Common Lisp Information
  ISI   will  collect  and  organize  public  domain  Common  Lisp  information
(currently over 20MB), maintain it in a library, update it as  new  information
is  generated  or  becomes available, and provide a dissemination mechanism for
it.

We will make use  of  the  ISI,  DARPA  funded,  Common  Lisp  Framework  (CLF)
persistent  object  oriented data base to store, house, locate and retrieve the
information types below, to link it  with  other  information  and  to  provide
release  and version management.  A network interface to the CLF data base will
be constructed to permit online retrieval of  the  archived  information.    In
addition,  CLF's  active  object  base  mechanisms will be used to construct an
automated dissemination facility (triggered  by  new  versions  of  the  stored
objects),  and  an automated postmaster which responds to stylized requests for
information that arrive via electronic mail.

The Network Services group in the Intelligent Systems Division of ISI  will  be
tasked  to  accomplish  much  of the library and document maintenance services.
This group currently exists and is chartered to provide expeditious, useful and
reliable  administrative computer and computer network support and service to a
number of clients of the Institute as well as to  funded  ISI  researchers  and
support staff.  This group has historically been able to bring to the community
a core staff of competent maintenance personnel who will be able to  coordinate
and  provide  the  expertise  required  to  help  act as a clearing house and a
communications link for the various library activities.

The specific information  and  procedures,  which  will  be  dealt  with  by  a
combination  of  Network Services and research staff overseeing this portion of
the overall effort, will include the following:


   1. Lists of Common Lisp users and implementers: It will be necessary to
      keep current lists of all Common Lisp user sites and implementers in
      order to ensure proper delivery of appropriate documents.  ISI  will
      undertake  this task, distributing the lists to parties that require
      them.

   2. The Common Lisp specification: The specification will constitute the
      baseline  document  which  at  all  times determines the Common Lisp
      language.  This baseline document will require strict  configuration
      management  to  determine  that it is kept appropriately up to date,
      while at the same time preserving the  integrity  of  the  language.
      Proposed  changes to this document will me developed, evaluated, and
      approved by the Common  Lisp  Technical  Committee.    All  approved
      changes will made by ISI staff.  To support this change process, ISI
      will collect and coordinate all requests for changes.  Such requests
      will  be  consolidated  and forwarded to the Technical Committee for
      their consideration.  ISI will collect  information  concerning  the
      actions  of  the  Technical  Committee  and  report  results back to
      interested  implementers  and  users.    Successful   use   of   the
      specification may require other explanatory documents for successful
      implementation.  ISI will develop and distribute such documents,  as
      well  as  the  specification  itself.   In addition, ISI will answer
      calls   and   queries   concerning   technical   issues    regarding
      interpretation and implementation of the Common Lisp specification.

   3. The  validation  tests  collected  under the previous task: When the
      validation tests have been completed, they  will  be  collected  and
      maintained  in  a  coordinated  and  accessible  database  for later
      perusal, modification, extension, and dissemination.   There  are  a
      number  of  different  mechanisms available which will be applicable
      for  this  chore  including:  shared  Electronic  Bulletin   Boards,
      extensive   and  moderated  mailing  lists  to  the  community,  and
      non-electronic newsletters.  The validation test suite will be  made
      available   to  whoever  requests  it  so  that  they  can  evaluate
      implementations of Common Lisp.  Instructions for use for use of the
      validation  test suite will be maintained and provided to interested
      parties.  Liaison hotlines will be manned to answer queries  and  to
      troubleshoot problems in proper application.

   4. An on-line version of the Common Lisp Manual: Maintaining an updated
      and  current  version  of  the  Common  Lisp  Manual  will  best  be
      accomplished by the Documentation Specialist in the Network Services
      group under the guidance of a  language  expert  and  the  Technical
      Committee.    This  document will be periodically changed to correct
      errors, extend or refine explanation, and to reflect changes to  the
      Common Lisp specification.  In addition, ISI will maintain a service
      to answer phone calls and written queries requesting  interpretation
      of  the  manual and to document requests for needed changes.  Change
      requests will be consolidated and prospective changes  that  respond
      to  legitimate  requests  will  be  drafted for consideration by the
      Technical Committee.   Approved  changes  will  be  implemented  and
      revised manuals will be sent to the Common Lisp community.

   5. Public  domain  implementations  of  Common  Lisp  or "Yellow Pages"
      additions to it: ISI will collect all implementations of Common Lisp
      compilers,   interpreters,   environments,  and  auxiliary  programs
      offered  by  the  community  for  use  by  other  interested  sites.
      Furthermore,  ISI  will  publish  a catalogue of these programs on a
      shared Electronic  Bulletin  Board.    Programs  will  be  available
      through  electronic  file  transfer  from  ISI or will be shipped to
      interested parties.  ISI will put parties who have implementation or
      utilization  questions  into contact with developers of the original
      software.

   6. Contributed  implementation  guidelines  and  hints:  Implementation
      guidelines  and helpful hints contributed by the user community will
      be maintained at ISI through a database  and  historical  record  of
      previous  Common  Lisp  implementations.    These archives will help
      vendors/contributors to easily access the information they  need  to
      get  themselves  on the right track for producing their own properly
      validated version  of  Common  Lisp.    An  automated  dissemination
      facility  and automated postmaster will take care of the bulk of the
      queries and problems.  Liaison hotline services and inhouse research
      staff available will resolve the exceptions.

   7. Electronic  forums on Common Lisp issues:  One of ISI's strong suits
      is the maintenance and moderating of electronic forums on a  variety
      of  issues.    A  Common Lisp bulletin board and/or a master mailing
      list of principals and players will be effectively managed from ISI.

   8. Proceedings of the  Common  Lisp  Executive  Committee:  Minutes  of
      Common   Lisp   Executive  Committee  meetings  will  be  collected,
      archived, transcribed, and selectively published by ISI.  This  will
      be  accomplished  through the established databases and tools of CLF
      as outlined above.


Since there will be a variety of requirements for distribution of requests  for
information,  for  test validation suites, and for other information, ISI is an
ideal site to center this general distribution activity.   Hopefully,  most  of
the  distributions  will occur electronically via the Internet, DDN and various
other Networks.  When this is not possible due to the electronic  isolation  of
some  vendor  or  site,  ISI will be able to transmit information in nearly any
format required including: Magnetic Tape (nearly any format or density), floppy
disk  and  paper/hardcopy.  ISI has a large mix of computers and peripherals as
well as a qualified and experienced Operations and  Systems  staff  which  will
allow  for  this variety of media production (as required).  Additionally, with
an Operations staff available 24 hours-a-day, 7 days-a- week,  researchers  can
be  assured maximum availability to the machine resources they might require or
expect to find.



2.3. Travel and Administrative Support for Convener
  Bob Mathis, an ISI employee, was chosen as the convener of  the  Common  Lisp
Standardization effort by the community. In this role, he will need substantial
administrative support  and  will  be  required  to  travel  extensively,  both
domestically   and  internationally,  to  attend  the  various  standardization
meetings  being  held  and  to  coordinate  this  standardization  effort  with
appropriate  technical  organizations.    We estimate that 50 to 80 days of his
time and extensive travel, especially internationally, will  be  required  each
year.

3. Current Activity and Plans



3.1. Validation Suites
  ISI  has  established  communication  with  the  various  Common Lisp vendors
regarding their previously "agreed upon" contributions  to  the  public  domain
validation  suite  and  set  in  motion  the process of collecting the existing
validation suites.  Most of the vendors are now in the  process  of  sanitizing
some  or  all  of their existing validation suites to remove proprietary and/or
non-portable material prior to delivering the source code to ISI.  Only a small
fraction of the expected contributions have actually been received.

  As  the  source  code  trickles  (and  then, hopefully, pours) in, it will be
necessary to "re-sanitize" them into some common  form.    Nearly  all  of  the
vendors  are  using  some form of test management software developed along with
their tests.  As one could expect, the nature of this test management is  quite
different from vendor to vendor.  Many are using some form of proprietary error
control mechanism, an area as yet not standardized in Common Lisp.

  From the technical (rather than administrative) point of view,  a  number  of
tasks must occur in order to successfully achieve the desired result of a truly
portable "universal" test suite for Common Lisp.  The primary task  is  one  of
understanding.    Anyone  who  has  ever  had to maintain or extend source code
created by another knows the inherent communication problem in reaching a  full
understanding  of  the source code.  In this particular case, we have added the
complexity of trying to understand source code which has not only been  written
by  a  number  of  others, but is written in and with the purpose of testing an
evolving language.  As anyone in the Common Lisp  development  community  could
attest, there are questions as to the exact meaning of a number of areas in the
current language specification (such as it is).

  Thus the task becomes one of heavy liaison between not only the ISI technical
personnel  responsible for the creation and maintenance of the validation suite
and the vendors, but also between the validation personnel  and  the  technical
steering  committee and the general implementors and development community.  It
would not be at all surprising if, due to questions raised during the  building
of  the validation suite, there arose (and subsequently were resolved) a number
of language design and implementation issues.

  Once each individual test (of which there may be hundreds  or  thousands  per
vendor  contribution)  has  been understood and validated as to being a test of
actual language features, and also of being a valid test of those features,  it
must  be  incorporated  into  some  kind  of  framework  under which all of the
validation testing occurs. A few vendors have offered to contribute  their  own
validation  managers,  but  as  of  this  time these have not been received for
evaluation.

  The most probable form of organization for the  test  suite  is  by  sections
numbered  as  in Steele's book, Common Lisp, the Language.  Thus, an individual
test must be exactly "locatable" as to the section in the book which  discusses
and  explains  the  feature  being  tested.    Because  this book has been much
clarified and revised by discussion over the network,  and,  in  fact,  because
this  process  is  ongoing, there is required the development of some "meshing"
between the validation effort and the continuing specification effort.

  Naturally, for any such validation suite to be  effective  it  must  be  both
broad in terms of language issues covered, and deep in the extent of exercising
the full range of functionality of each language feature.  The ideal validation
would  exhaustively attempt every type of operation with every type of operand,
both legal and intentionally illegal.  It is just as  important  to  test  that
error  conditions  are  detected  and  reported  correctly as it is to test for
simply correct functionality of legal expressions.

  One of the initial hurdles that  must  be  overcome  in  the  creation  of  a
validation test management program is the nature of the detection and reporting
of exception or error conditions.  The common consensus is to  use  "the  error
handling  mechanism"  of  the  language.    Indeed, all of the vendors who have
offered to contribute test management software, use the language error handling
features.    Unfortunately  at  present  there  is  no  standard  for the exact
definition and handling  of  exception  conditions,  and  so  each  vendor  has
implemented   their   own   form   of   error  handling.    Either  a  portable
non-exception-based test control  mechanism  must  be  devised,  or  the  error
handling features must be put in place in the language specification so that it
can be relied upon during validation.

  During this initial phase of  soliciting  code  contributions  from  vendors'
existing  validation  suites,  ISI  will also be evaluating those contributions
against the idealized validation  suite  described  above.    As  more  vendors
specify  the  exact nature of their test contributions, ISI can use this survey
to identify the holes in the composite suite  formed  from  the  contributions.
Several  vendors have indicated that they would be willing to create wholly new
validation tests for specific areas  of  the  Common  Lisp  Manual.    We  will
coordinate   these   vendor  activities  to  maximize  coverage  and  eliminate
duplication.  Once a validation framework is decided upon  for  the  individual
tests,  we  can  then  specify  the  format for all these new tests so that the
effort of integrating them into the full validation suite is minimal.

  Besides these  mostly  technical  tasks,  there  is  a  large  administrative
responsibility  in  the ongoing communication with validation contributors, the
technical  committees,  the  language  developers  and  the  Common  Lisp  user
communities.    Not  all  developers  have  network access in a form that makes
distribution over the network possible.  In the interest of  full  support  for
all  vendors  and developers, the public domain validation suite should be made
available on tape and in other ways that  would  facilitate  its  distribution.
When  revisions  and/or  additions  are  made  to  the  validation  suite,  the
interested parties must be notified.  Undoubtably, when a validation  suite  is
made  public domain, there will not be complete agreement as to the validity of
the test itself.  This could be despite  approval  by  the  various  committees
involved.    Especially  in  its initial incarnations there will certainly be a
succession of incremental revisions designed to correct flaws in the individual
tests of the validation suite.

  To  maintain  complete impartiality, and to provide for a uniform standard of
reporting, when a vendor desires an official validation rating ISI will conduct
the  test  of  the  implementation.  By going to the vendor's site we avoid the
need to send hardware to ISI, and all  the  subsequent  problems  that  revolve
around  that.    A  validation  team would actually use the validation suite to
verify  the  implementation's  correctness.    The  results  of  this  official
validation  run  would  be  collected by the validation team, put into a formal
report in a standard format, and  forwarded  to  the  Technical  Committee  for
evaluation.    By  using this more formal approach, it will be possible for the
Technical  Committee  to  objectively  compare  different  implementations  for
determining the level of compliance with the full Common Lisp standard.



3.2. Keeping Track of Decision Making and the "Why's"
  Another  area of ISI's participation as an administrative body for the Common
Lisp community is technical  record  keeping.    Currently,  a  great  deal  of
discussion  occurs  on  the network regarding specific details of the language,
its specification and implementation.  Often an apparent consensus  is  reached
to  alter  or  drop  "old" language features, or to add entirely new ones.  Yet
there does not appear to be any official statement from  a  technical  steering
committee regarding any decisions to accept proposals.  And too, often there is
no formal proposal, but simply a semblance of agreement amongst those arguing a
point.

  It is not unusual to see something like:

   - As  I  recall, we discussed point "x" several months ago, but I can't
     seem to recall what we decided...

  Clearly, this causes repetition of whole discussions, and in  general  throws
rocks  in what is already an unpaved road.  Ideally there should be some method
of classifying both the content and nature of each message,  as  well  as  some
form  of  discussion  tracing,  all  built into a kind of archiving methodology
which provides  querying  and  possibly  browsing  facilities.    Once  such  a
record-keeping  system  is  in  place,  further  discussion participants can be
advised as  to  the  classification  system  so  that  they  can  indicate  the
classification  for  their  messages.    In  the meantime, and especially while
developing the classification method, each message will have to  be  classified
at ISI.

  During  these discussions, many valid points are raised and, often, resolved.
A great deal of the philosophy of the language design  is  contained  in  these
messages.  There are a number of ways in which these messages can be classified
and organized.  There are two main areas of classification:

   1. The "type" of message.

   2. The topic(s) of the message.

  While it would be better  if  a  message  contained  exactly  one  topic,  in
practice  even  one  "topic"  easily branches off into several related subjects
which wouldn't be appropriate to separate messages.  Thus each message would be
classified  by  a  list  of  topics  discussed.  Often this would be a specific
Common Lisp function or feature.  At  other  times  it  might  discuss  a  more
general  area,  such  as  error  handling, or string representation.  It may be
feasible to assume that there is a finite set of topics  into  which  the  more
specific  topics can fit.  For example, "string representation" might fit under
"strings", as would discussions about specific string functions.

  When classifying by topic (and especially when considering a  finite  set  of
topics)  there  is  a useful guide -- Steele's book, Common Lisp, the Language.
The list of finite topics could well be the section numbers in  that  book,  so
that when discussing a particular topic or topics, the message sender must know
what sections of the book contains the material which he  has  some  discussion
concerning.    This  has  the added advantage of making the message sender more
aware of the published information regarding his interest.  Thus, if this  were
the  decided-upon classification method, each message on the network concerning
Common Lisp would have a list of section numbers, put into the message  by  the
sender.

  The  querying  or browsing facility mentioned above would then allow the user
to view (and summarize, count, etc.) the messages regarding  a  certain  topic.
Naturally  these messages would appear in temporal sequence.  A typical request
might be to summarize (i.e. print the sender, date  and  discussion  topic)  of
each message on a given set of topics within a certain time period.  This would
help to locate a specific message by jogging the searchers memory.

  The other main form of classification is by "type" of messages.  A  key  type
might  be  "Ratified  Decision", or some similar type.  This would be a message
(from the steering committee) officially announcing some change to the language
specification.  Like any other message, it would be classified by topic(s).  By
using a type-classification as well, a user could  now  search  for  "decisions
made  about  strings between February and June of 1986".  If one only cared for
specification changes, this would be an efficient means of getting the  concise
information needed.

  Other  types can be suggested, such as "Start of new discussion", which would
indicate the first message in a new  chain  of  discussion.  Or  "Clarification
request",   "Proposal",   etc.    Undoubtably  the  process  of  building  this
record-keeping mechanism will bring to light the required set of message types.

  Another valuable means of organizing these messages is by discussion tracing.
By  this is meant the linking, in temporal sequence, of each message in a given
discussion, from the inception of the discussion through to its resolution.  In
a browsing mode, the user might be able to start from any message and trace the
discussion chain forward and backward to get the full set of things  considered
when  resolving the discussion.  This is especially useful because a great deal
of traffic is currently generated in repeating discussions resolved earlier.

  Probably other ways of organizing the Common Lisp  discussions  will  present
themselves  as  the  task  progresses.    By  using  the  Common Lisp Framework
developed at ISI, each message can be treated as a  unique  object,  and  given
properties  (such as type and topics) which relate the messages one to another.
The facilities provided under CLF  allow  for  complex  searching  to  be  done
amongst these objects.  A stylized querying language and a reporting method can
be created so that network users can access the CLF Common Lisp data base to do
the  research  they  need  to  more  optimally  interact  with  the development
community.

4. Milestones

   - 3 Months

        * Build a Validation Manager to  invoke  and  collect  results  of
          individual tests.

        * Coordinate the focus of vendors contributing to Validation Suite
          to maximize coverage.

   - 6 Months

        * Integrate initial contributed Validation Suites into Library

             1. Homogenize the tests so that they are included in a common
                way  and  report their results in a common way (currently,
                each vendor has their own conventions).

             2. Eliminate duplicate tests.

             3. Determine the areas of coverage and the degree of coverage
                within those areas.

   - 9 Months

        * For each test in the initial contributed Validation Suites:

             1. Determine validity of test

                   - Revise or remove incorrect tests

                   - Forward questionable ones to Technical Committee

             2. Place test in appropriate portion of Validation Suite

             3. Catalog  it  by its contributor, the test it performs, and
                its time of contribution

   - 12 Months

        * Make information in the Common  Lisp  Library  accessible.  This
          information includes:

             1. The Common Lisp specification.

             2. The Public Domain Validation Suite.

             3. An on-line version of the Common Lisp Manual.

             4. Public  Domain  Implementations  of Common Lisp or "Yellow
                Pages" additions to it.

             5. Contributed implementation guidelines and hints.

             6. Electronic forums on Common Lisp issues.

             7. The Proceedings of the Common Lisp Executive Committee.

   - 15 Months

        * Design  procedures  by  which  on-site  validation   of   vendor
          implementation  of  Common  Lisp will be carried out. Coordinate
          with vendors and Common Lisp community.

   - 18 Months

        * Automated Postmaster and Dissemination mechanism for Common Lisp
          information

        * Begin conducting on-site validation of vendor implementations

5. Computing Resources
  ISI   does   not   require  any  additional  (DARPA  purchased)  Common  Lisp
Workstations for this project.  The necessary  workstations  will  be  supplied
from  a  pool of grant Common Lisp Workstations given to ISI by Hewlett Packard
and Texas Instruments in recognition of DARPA sponsored work  in  the  Software
Sciences Division on FSD.

  Operations  and  maintenance  costs  of  these workstations, however, must be
included in the budget for this effort.

-------